5.2 Test Result Evaluation 5.1 Test Case Generation

the test classes a collection of strategies in order to derive test input data. These strategies will be formalized such that they can be put into a rule base for a term rewriting system. Additionally, diierent tactics will be developed to selectively apply these strategies to diierent parts of the speciication. The result of such an automatic process will be a diierent collection of test classes, which will give a much ner granularity of cases of interest than described by the DNF test classes currently produced, which only reeect the control structure of the system. Acknowledgments Parts of the work presented here have been funded by the Technologiestiftung Schleswig{Holstein within the CATI project. We would like also to thank Jan Meyer and Martin Schmitz for their contributions to the case study. in order to carry out the transformations described in Section 5.1. This gure might improve somewhat, once the testers get more familiarwith the methods and tools. Demonstration of traceability and completeness against the speciication has been easier, if not accepted by construction. The main advantages were observed during test result evaluation, where the task carried out manually over several days reduced to about one hour of automatic execution. Evaluating regression tests shows a similar relation, but the manual evaluation had the old test output as a reference to be compared to, manually again. Preparation of the test environment had been somewhat more expensive because the Event/Action Transformer had to be implemented and tested. But this is a xed eeort carried out once at the beginning of the testing process. One question arising naturally is how possible errors in the test environment might lead to correct programs to be claimed faulty or incorrect programs claimed to be correct. As far as the xed part of the test environment is concerned , this will be tested more and more during its application, leading to a speciic degree of conndence. A weak point is the Event/Action Transformer which has to be written manually by the testers. As long as this is developed independently of the application code, a redundant parallel development reduces the risk of errors compensating each other in application code and the test environment and therefore left undetected. The remaining risk of errors in the test environment, however, must be compared to the risk of errors arising during manual routine work, especially when carried out again during regression test. 6 …